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Introduction 



Politicians, policymakers, and private 
foundations have united in recent years 
around achieving a common goal: 
college for all. As President Barack 
Obama pledged in his first speech to 
a joint session of Congress in February 
2009: “We will provide the support 
necessary for you to complete college 
and meet a new goal: by 2020, America 
will once again have the highest 
proportion of college graduates in 
the world.” At the beginning of the 
21st century, increasing and ensuring 
individual access to college presents 
itself not just as a moral imperative, but 
an economic necessity. As employment 
opportunities in manufacturing continue 
to grow scarcer in the United States, 
both individual and national global 
economic competitiveness requires 
mastery of what many commentators 
have termed “21st century skills.” 

These skills, generally thought uniformly 
taught at U.S. colleges and universities, 
are defined as including critical thinking, 



complex reasoning, and written 
communication. But what if sending 
students to college did not necessarily 
ensure that much was learned once 
there? What if at the beginning of 
the 21st century many colleges and 
universities were not focused primarily 
on undergraduate learning, but instead 
had become distracted by other 
institutional functions and goals? 

We have systematically investigated 
the state of undergradua te, learn ing: 
in contemporary colleges and 
universities. Following several thousand 
traditional-age students as they enrolled 
in coursework from Fall 2005 to Spring 
2009, across a wide range of four-year 
colleges and universities, we found a set 
of conditions suggesting that something 
indeed is seriously amiss in U.S. higher 
education. In Academically Adriji: 
Limited Learning on College Campuses 
(University of Chicago Press, 2011), 
we have described some of these 



disturbing conditions and documented 
the extent to which many students 
show little if any growth over the 
first two years of college in their 
ability to perform tasks requiring 
critical thinking, complex reasoning, 
and written communication as 
measured by the Collegiate Learning 
Assessment (CLA; for more 
information on the CLA see the 
Methodological Appendix). This 
report extends findings reported in 
our recent book to document the 
rate of growth on the CLA for the 
full four years of college, academic 
practices associated with improved 
student performance, as well as 
differences across individuals and 
institutions in the level of learning. 
Moreover, we present recommenda- 
tions for policymakers, institutions, 
and practitioners to consider for 
improving undergraduate learning 
at U.S. colleges and universities. 





Limited Academic 
Engagement and 
Learning Outcomes 



► 



While higher education is expected 
to accomplish many tasks, existing 
organizational cultures and 
practices too often do not prioritize 
undergraduate learning. Large 
numbers of college students report 
that they experience only limited 
academic demands and invest only 
limited effort in their academic 
endeavors. Findings from our 
study document that: 




■ Large numbers of students 
report that they enroll in courses 
that do not require either substantial 
writing or reading assignments. 

In a typical semester, 32 percent 
did not take any courses with more 
than 40 pages of reading per week 
and 50 percent did not take a single 
course in which they wrote more 
than 20 pages over the course of 
the semester. A quarter of students 



experienced neither of those course 
requirements in a typical semester. 
Over their entire four years of 
college coursework, 50 percent of 
students reported that they had taken 
five or fewer courses that required 
20 pages of writing over the course 
of the semester, and 20 percent of 
students reported that they had taken 
five or fewer courses that required 
40 pages of reading per week. 



Figure 1. Reading and writing course requirements 
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■ On average, students in a typical 
semester spend only between 12 and 14 
hours per week studying (approximately 
50 percent less time than full-time 
college students did a few decades ago, 
according to recent research by labor 
economists Phillip Babcock and Mindy 
Marks published in the Review of 
Economics and Statistics). Combining 
the hours spent studying with the hours 
spent in classes and labs, students spend 





less than one- fifth (16 percent) of their 
time each week on academic pursuits. 



■ When studying, students attending 
four-year colleges on average spend 
about one-third of their time studying 
with peers in social settings that are not 
generally conducive to learning 
Excluding studying with peers from our 
calculations, college students on average 
spend only between 8 and 9 hours per 




week studying alone. More troubling 
still, 35 percent of students at four-year 
colleges report that they spend five or 
fewer hours per week studying alone. 

■ Students interact with their 
professors outside college classrooms 
rarely (on average only monthly), if 
ever (9 percent of students never meet 
with faculty outside the classroom 
in a typical semester). 



Figure 2. Student time use 
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Greek and other clubs 
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Socializing, recreating, 
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Note: Based on Spring 2007 survey. 

Percentages are based on 168 hours - i.e., full seven-day week. 








Limited Academic 
Engagement and 
Learning Outcomes 

Continued 




Our findings on limited collegiate 
academic investment and curricular 
demands replicate findings identified 
by the National Survey of Student 
Engagement (NSSE), which over the 
past decade has surveyed more than 
2 million students at more than 1 ,000 
colleges and universities. Among 
other results, NSSE reveals a limited 
amount of time students spend 
studying (13 to 14 hours for full-time 
students, 9 to 1 0 hours for part-time 
students) and meager writing 
requirements found in contemporary 
undergraduate coursework (for 
example, 51 percent of college seniors 
reported that they had not written 
a paper during the current academic 
year that was 20 or more pages long; 
even at the top 10 percent of schools 
in the NSSE study, 33 percent of 
college seniors reported that they 
had not written a paper of this length 
during their last year in college). 




Given the limited academic engage- 
ment shown by many students, it is 
not surprising that we find that gains 
in student performance are disturbingly 
low. We find in our study that on 
average, gains in critical thinking, 
complex reasoning, and writing 
skills (i.e., general collegiate skills) 
are either exceedingly small or 
empirically non-existent for a large 
proportion of students: 

■ Forty-five percent of students 
did not demonstrate any significant 
improvement in learning, as meas- 
ured by CLA performance, during 
their first two years of college. 

■ Considering all four years of 
college, we find that 36 percent 
of students did not demonstrate 
any significant improvement in 
learning, as measured by CLA 
performance. 





■ On average, students improved 
performance on the CLA by only 
0.18 standard deviations over the 
first two years of college and 0.47 
standard deviations for the full four 
years of college. Students who 
scored at the 50th percentile of 
students in their entering freshman 
cohort would have moved up 
only to the 68th percentile after 
four years of college (if, when 
graduating college, the students 
retook the test with a new cohort 
of entering freshmen). In an 
extensive review of the literature 
presented in How College Affects 
Students , Ernest Pascarella and 
Patrick Terenzini estimated that 
students in the 1980s learned at 
twice the current rate — seniors 
at that time had an advantage 
of one standard deviation over 
freshmen in critical thinking. 



If students’ self-reports of their own 
academic engagement were not so 
limited, one might be tempted to 
dismiss these findings as a method- 
ological artifact associated with 
a limitation of the essay-based, 
open-ended assessment approach 
used by the CLA. Related work 
conducted by a research team led 
by Charles Blaich at the Center of 
Inquiry in the Liberal Arts at Wabash 
College also puts such objections to 
rest. While we found that students 
on average only gained 0.47 standard 
deviations on the CLA between 
their fall freshman and spring senior 
semesters, Blaich found that students 
in participating colleges he studied 
gained only 0.44 on an alternative 
close-ended, multiple choice 
assessment indicator of critical 
thinking and complex reasoning 
(ACT’s Collegiate Assessment of 
Academic Proficiency). 



Academic Factors 
Associated 
with Learning 



When exploring collegiate experiences 
associated with improved student 
learning, we found consistent evidence 
that certain factors were beneficial 
to growth in CLA performance and 
others were not. In general, our 
findings suggested that educational 
practices associated with academic 
rigor improved student performance, 
while collegiate experiences associated 
with social engagement did not. 







Figure 3. Predicted 2007 CLA scores by faculty 
expectations and reading/writing course requirements 
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Note: Predictions based on a model estimating 2007 CLA scores while 
controlling for 2005 CLA scores, student characteristics, and institutions attended. 



Specifically, our results identified 
the following measures associated 
with academic rigor that were 
conducive to improved student 
performance on the CLA: 

■ Students who spent more hours 
studying alone had greater gains 

on the CLA. 

■ Students who took courses 
requiring both significant reading 
(more than 40 pages per week) 
and writing (more than 20 pages 
over the course of the semester) 
had higher rates of learning. 

■ Students reporting faculty 
with high expectations at their 
institutions had higher rates 
of learning. 

■ Students who had more 
advanced coursework in high 
school had greater subsequent 
gains on the CLA in college. 





While educational practices associated 
with academic rigor facilitated learning, 
measures associated with student social 
engagement did not: 

■ Students who spent more 
hours studying with peers showed 
diminishing growth on the CLA. 

■ Students who spent greater 
hours in fraternities and sororities 
had decreased rates of learning. 

■ Other student activities, such 
as working on or off campus, 
participating in campus clubs/ 
organizations, and volunteering, 
were not related to learning. 

The extent to which our findings 
highlight the importance of academic 
rigor over social engagement for 
learning should serve as a cautionary 
signal to colleges that have emphasized 
the latter in efforts to increase student 
retention in higher education. These 
results, however, do not suggest that 
all forms of group-learning experiences 
should be abandoned. Group learning 
in specifically structured contexts, 
such as within learning communities 
or particular majors, may facilitate 
students’ mastery of course material 
and development of general skills. 
Further research is needed to assess 
how specific types of group-centered 
activities are related to objective 
measures of student learning. 





Variation 
Across Institutions 
and Individuals 



► 



While the average trends in our data 
indicate that too many students are 
embedded in institutions that place 
very limited academic demands 
on them and that limited learning 
occurs for all too many students 
during college, there is notable 
variation across students as well 
as across institutions. In our study, 
we found many high-performing 
students from all socio-economic 



backgrounds and racial/ ethnic 
groups, as well as students with 
different levels of academic 
preparation, who improved their 
performance on the CLA at 
impressive rates while enrolled in 
college. In virtually every college 
examined we found students who 
were devoting themselves to their 
studies and learning at rates 
substantially above the average. 




There is notable variation in 
experiences and outcomes across 
institutions. Students attending 
certain institutions have more 
beneficial college experiences 
(in terms of reading/ writing 
requirements, hours studying, 
and high faculty expectations) and 
demonstrate significantly higher 
gains in critical thinking, complex 
reasoning, and writing skills over 
time, even after we control for 
students’ individual characteristics. 



■ Colleges and universities make 
a difference in improving student 
performance. Over four years of 
college, we find that 23 percent 
of variation in CLA performance 
occurs across institutions. 



Many institutional factors (such as 
institutional type, size, sector, and 
resources) could potentially be 
related to variation in student 





performance. We focused on 
one institutional variable that past 
sociological research has shown 
can have a strong relationship 
to learning: peer composition. 

In particular, we measured the 
selectivity of entering freshmen 
at an institution and examined the 
extent to which this measure of 
peer characteristics was associated 
with variation in rates of learning 
over and above a student’s own 
individual characteristics. 

■ Institutional differences 
associated with student selectivity 
are related to increased growth in 
CLA performance over four years 
of college. 

Selectivity of entering freshmen 
is a measure that likely taps multiple 
sources of institutional variation, 
including differences in peer 
orientations and student behaviors 



with respect to academic endeavors, 
variation in curricular demands, 
as well as variation in faculty 
expectations. 

There is more variation within 
institutions than across institutions. 
Although institutional selectivity 
is related to students’ experiences 
and growth in the CLA, high- and 
low-performing students can be 



found at each institution and within 
each level of selectivity. 

■ Students demonstrating 
substantial learning (e.g., the top 10 
percent of the CLA growth distribu- 
tion) are found at each institution. 




institutions one standard deviation above the mean; "low selectivity" represents institutions 



one standard deviation below the mean. 






Variation 
Across Institutions 
and Individuals 



Continued 



Learning in higher education is 
characterized by persisting and/or 
growing inequality . There are 
significant differences in critical 
thinking, complex reasoning, and 
writing skills across students from 
different family backgrounds and 
racial/ ethnic groups: 

■ Students who come from families 
with different levels of parental 
education enter college with different 
levels of performance but learn at 
approximately equivalent rates while 
attending four-year colleges. 

■ Black-white gaps in student test 
score performance, however, increase 
in magnitude over four years of 
college. African American students 
improve their CLA performance 

at lower levels than white students 
during four years of college. 



Findings of persisting and/ or growing 
inequality are consistent with research 
on K-12 education. Gaps in CLA 
performance across students from 
different family backgrounds in our 
study can be explained by the varying 
levels of academic preparation with 
which students enter higher education. 
Mirroring results from K-12 research, 
most but not all of the black-white 
gap can be explained by measured 
individual and institutional characteris- 
tics. Approximately two-thirds of 
the black- white gap in CLA growth in 
our study is accounted for by academic 
preparation, college experiences, and 
institutions attended. 

There is notable variation in academic 
experiences and outcomes across fields of 
study. Variation by field of study may 
reflect a range of different factors, 
including differences in curriculum as 
well as differences in academic aptitude 
and orientations of students who 
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have chosen particular majors. While 
appreciating the diverse causes of 
differences by field of study, we 
observed several patterns in our data: 

■ Students majoring in traditional 
liberal-arts fields, including social 
science, humanities, natural science, and 
mathematics, demonstrated significantly 
higher gains in critical thinking, complex 
reasoning, and writing skills over time 
than students in other fields of study 

■ Students majoring in business, 
education, social work, and communica- 
tions had the lowest measurable gains. 

■ Greater gains in liberal-arts fields 
are at least in part related to faculty 
requiring higher levels of reading 
and writing and students spending 
more time studying. 



Higher performance of students 
in liberal- arts fields may not be 
surprising given the focus of the 
CLA on general skills, including critical 
thinking, analytical reasoning, and 
writing. Reported findings do not 
preclude the possibility that students 
in other fields are developing 
subject- specific or occupationally- 
relevant skills. These patterns, 
however, do challenge the assumption 
that focusing on field- specific 
knowledge will inevitably lead to 
improvement in general skills. 

Differences in student outcomes both 
across and within higher education 
institutions highlight the extent to 
which colleges seeking to improve 
student learning can gain as many 
insights by looking within for 
institutional strengths and weaknesses 
as looking outside to exemplary 
institutions. NSSE’s work on Project 
DEEP (Documenting Effective 
Educational Practice) and the efforts 



of the Association of American 
Colleges and Universities (AAC&U) 
to develop principles of effective 
practices, such as those outlined in 
General Education & Liberal Learning: 
Principles of Effective Practice , provide at 
least two recent examples of on-going 
efforts to improve undergraduate 
academic experiences and outcomes. 








Policy 

Recommendations 




In recent years, discourse around 
higher education has increasingly 
focused on improving college access, 
retention, and completion. Our 
research, however, highlights the need 
for policymakers and practitioners 
to focus as much attention on 
improving student learning as has 
been demonstrated in recent years 
toward these other laudatory goals. 
There are no simple solutions to the 




problem of limited learning on 
college campuses. Improving 
learning outcomes in undergraduate 
education will require the commit- 
ment and dedication of a diverse set 
of individuals and institutions. One 
cannot mandate learning through the 
imposition of increased regulation; 
rather, for change to occur, various 
entities and actors will have to 
demonstrate in their behaviors and 




actions a deep commitment to these 
goals and a willingness to take personal 
and institutional responsibility for their 
successful achievement. 

Before identifying policies and 
practices that we believe will lead to 
improvement in student learning, we 
explicitly note here our reservations 
to one proposed remedy that 
has received much attention. 

In our estimation: 

■ Externally imposed accountability 
systems would be counterproductive. 

Our findings are disturbing enough 
that many well-intentioned legislators 
might be easily tempted to call for the 
imposition of a federally mandated 
accountability system for higher 
education institutions, similar to 
those recently introduced in K-12 
education. While such sentiments 
would be understandable given our 
findings on the state of undergradu- 
ate learning, we believe that such 
efforts at this time would be counter- 
productive. Our current ability to 
measure accurately diverse forms of 
students’ general and subject- specific 
performance is still in the early stages 
of development. While we support 
the promotion of increased personal 
and institutional responsibility to 
address this issue, the imposition 
of federally mandated regulatory 
schemas would likely have unintended 
negative consequences that would 
far outweigh any demonstrated 








improvements. Efforts to mandate 
the use of specific measures for 
accountability purposes (as opposed 
to promoting their use for research 
and internal formative assessment tied 
to improving instruction) would likely 
be counterproductive at this time. 

In the changing economic and global 
context facing contemporary college 
graduates, and our commitment to 




the development of an educated 
citizenry capable of contributing 
constructively to a democratic society, 
we offer several recommendations 
as next steps for improving 
collegiate learning. 

K-12 System Improvement 

■ Improved elementary- and 
secondary-school student preparation. 

Given that students who are better 



prepared upon matriculation into 
college demonstrate increased 
learning and better performance 
in higher education, there must 
be continued concerted efforts to 
improve academic preparation. 
Current endeavors to increase 
achievement in K-12 and to align 
postsecondary curricula with higher 
education deserve careful attention 
and need to be supplemented with 
efforts to help students develop 
mature values and behaviors 
conducive to learning. 

Higher Education Institutional 
Recommendations 

■ Strong leadership in higher 
education to foster an institution-wide 
culture of learning. Institutional 
leaders and administrators need to take 
responsibility for modifying existing 
internal institutional incentives and 
practices to develop — and articulate 
a commitment to embracing — 
organizational cultures that prioritize 
undergraduate learning and student 
assessment. College and university 
trustees, regents, and overseers must 
work to ensure that institutional 
leaders and administrators prioritize 
undergraduate learning as an organiza- 
tional goal. Institutional leaders 
and administrators should be held 
accountable by these existing 
governance mechanisms, which 
should require internal reporting 
on measurable student learning 
outcomes, expenditures devoted 








Policy 

Recommendations 

Continued 



to instructional purposes, and 
institutional activities devoted to 
improving instruction. 

■ Enhanced curriculum and 
instruction associated with academic 
rigor. More rigorous, appropriately 
demanding course requirements and 
standards must be put in place to ensure 
the development of critical thinking, 
complex reasoning, and written 



communication skills (i.e., increased 
academic assignments requiring greater 
student effort, adequate student reading 
and writing, and high expectations by 
faculty). Moreover, if active/collabora- 
tive learning approaches outside the 
classroom are adopted as instructional 
models, these models must be 
specifically structured and carefully 
assessed to ensure that adequate 
academic development is occurring. 



Faculty must take collective and 
individual responsibility for ensuring 
that coursework offered at their institu- 
tions is both rigorous and demanding. 

• Faculty should have high standards 
and expectations for their students 
and should be adequately trained and 
supported by their institutions to serve 
as effective instructors. 
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• Students must take responsibility 
for embracing the academic 
opportunities they have been 
afforded. Colleges and universities 
must promote organizational 
cultures that encourage and require 
students to demonstrate serious 
commitment to academic endeavors 
and to take responsibility for devoting 
sufficient time to their studies. 

■ Increased measurement of 
student learning. Colleges and 
universities should consistently 
collect diverse, comprehensive 
sources of evaluation and assessment 
data to improve instruction and 
student learning on an ongoing basis. 
Given the great variation in student 
performance within institutions in 
our study, all schools would benefit 
from developing internal organiza- 
tional mechanisms to identify the 
strengths and weaknesses of their 
respective curricular programs in 




order to engage in ongoing instruc- 
tional improvement efforts that 
are informed by assessment data. 
Institutions can learn from and 
build on the efforts of colleges 
and universities currently engaging 
in assessment of learning for 
institutional improvement, such as 
those participating in the Voluntary 
System of Accountability or the 
New England Consortium on 
Assessment and Student Learning. 

System Level Actions 

■ Reallocation of government 
resources to improve undergraduate 
learning. While public investment in 
research is a worthy end, federal and 
state government grant programs tied 
to improvement of undergraduate 
learning and its assessment on both 
institutional and individual levels 
would encourage the development 
of policies and practices to enhance 
undergraduate educational outcomes. 



■ Enhanced research infrastructure 
for measuring and understanding 
student learning in higher education. 

In addition to the recommendations 
for practitioners and policymakers 
identified above, our research 
identifies the importance of 
embedding longitudinal measurement 
of collegiate learning outcomes in 
future federally organized national 
probability studies. Such measures 
would greatly enhance the available 
research infrastructure to allow future 
social science researchers to better 
identify determinants of collegiate 
learning. Moreover, limited place 
randomized field trials of promising 
curricular and instructional models 
would generate increased knowledge 
of effective programmatic interven- 
tions to guide policy and practice. 



Methodological 

Appendix 



participated in the project. Most 
of the reported regression analyses 
are based on the first two years 
of data, including 2,322 students 
attending 24 institutions. The sample 
includes a range of institutions from 
all four regions of the country, of 
varying sizes, selectivity, and missions, 
from liberal arts colleges and large 
research institutions to a number 
of Historically Black Colleges and 
Universities (HBCUs) and Hispanic 
Serving Institutions (HSIs). 



Determinants of College Learning 
( D CL ) Dataset 

Presented analyses are based on the 
Determinants of College Learning 
(DCL) dataset, which was developed 
in partnership with the Council for 
Aid to Education (CAE). The CAE 
initiated the Collegiate Learning 
Assessment (CLA) Longitudinal 
Project in Fall 2005, administering a 
short survey and the CLA instrument 



to a sample of traditional- age 
freshmen at four-year institutions. 
The same students were contacted 
for the sophomore-year follow-up 
in Spring 2007 and senior-year 
follow-up in Spring 2009. Some 
institutions and students participated 
in all three waves of the study while 
others participated in only some of 
the waves. In total, more than 3,000 
students at 29 four-year institutions 



In addition to completing the 
CLA, in 2007 and 2009 students 
completed a brief survey covering 
two main areas: college experiences , 
including questions regarding 
academic experiences, extracurricular 
involvement, employment, faculty 
and peer climates, college financing, 
and college major; and background 
information , including questions 
regarding sociodemographic 
characteristics, high schools attended, 
academic preparation, and educa- 
tional and occupational aspirations. 

Logistic and resource constraints 
required reliance on participating 
institutions to implement appropriate 
random sampling and retention 
strategies. We thoroughly 
investigated the extent to which 
students in our sample were indeed 
representative of students from these 
institutions as well as U.S. higher 
education more broadly. On most 
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measures examined, students in the 
DCL dataset appeared reasonably 
representative of traditional-age 
undergraduates in four-year institu- 
tions and the colleges and universities 
they attended resembled four-year 
institutions nationwide. DCL 
students’ racial/ ethnic and family 
backgrounds, as well as their English- 
language background and high school 
grades, tracked well with national 
statistics (see Table 1). Moreover, 
four-year colleges and universities 
in the DCL sample have a similar 
proportion of white students and a 
similar level of academic preparation 
as do four-year institutions in general 
(see Table 2). Indeed, the 25th and 
75th SAT percentiles of the entering 
student body at the DCL institutions 
and four-year institutions nationwide 
are virtually identical. However, 
as a result likely of the voluntary 
participation required in our study, 
our sample did have fewer men as 
well as a smaller number of students 
of lower scholastic ability as measured 
by standardized tests (e.g., students’ 
combined scores at the 25th 
percentile of the SAT were higher in 
our sample than at DCL institutions 
or four-year institutions nation-wide). 
Consequently, we believe that any 
biases introduced into our analysis 
by the sampling procedures used in 
the study are likely to be in the 
direction of leading us to over- 
estimate students’ positive educational 
experiences and institutional success. 



Table 1. Student characteristics and experiences in the DCL and BPS samples 





DCL Sample 3 


BPS Sample 11 




Mean 


Mean 


Background Characteristics 






Race/ethnicity 






White 


0.64 


0.70 


African-American 


0.15 


0.09 


Hispanic 


0.05 


0.10 


Asian 


0.10 


0.05 


Other 


0.05 


0.06 


English not primary language 


0.13 


0.10 


Parental education 






High school or less 


0.14 


0.23 


Some college 


0.21 


0.18 


Bachelor's degree 


0.29 


0.29 


Graduate/professional degree 0.36 


0.30 


High school GPA 






D or lower 


0.00 


0.00 


C-/C 


0.00 


0.01 


C/B- 


0.03 


0.05 


B-/B 


0.10 


0.10 


B/A- 


0.38 


0.35 


A-/A 


0.48 


0.49 


College Experiences 






Employment 






Not working 


0.35 


0.29 


Average # of hours working 


12.48 


20.50 


(if employed) 






Studying with peers 0 






Never 


0.23 


0.24 


Sometimes 


0.61 


0.58 


Often 


0.16 


0.18 


Meeting with faculty outside of class 0 




Never 


0.09 


0.09 


Sometimes 


0.60 


0.55 


Often 


0.32 


0.36 



a The DCL sample includes students who participated in the 2005 and 2007 waves. This is the sample on 
which most of the reported analyses are based. 

b Beginning Postsecondary Students (BPS) Longitudinal Study, 2003-2004 cohort. The sample is restricted 
to students who entered four-year institutions and were 19 years of age or younger as of 12/31/2003. 
College experiences are based onthe 2006 survey and thus typically reference students' junior year. 

c These categories represent BPS coding. DCL questions were mapped as closely as possible to these 
categories (for hours studying with peers: 0 - never, 1 -5 hours - sometimes, above 5 hours - often; for times 
met with faculty: 0 - never, 1 -4 times - sometimes, more than 4 times - often). 
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Table 2. Institutional characteristics of colleges and universities in the DCL sample compared to IPEDS reports 





Our Sample: DCL 
Schools 3 
Mean 


IPEDS: DCL 
Schools Only b 
Mean 


IPEDS: All Four-Year 
Institutions 11 
Mean 


Institutional Demographics 








Total Score 2005 0.37 

Total Score 2007 0.59 

Difference between 2007 and 2005 Total Score 0.19 


0.46 

0.61 

0.14 


0.45 

0.59 

0.13 


% Male 


0.37 


0.46 


0.45 


% White 


0.59 


0.61 


0.59 


% African-American 


0.19 


0.14 


0.13 


% Hispanic 


0.05 


0.08 


0.13 


% Asian 


0.11 


0.10 


0.06 


% Other 0 


0.05 


0.07 


0.09 


Institutional Selectivity 








SAT, 25th percentile 


1052.83 


995.15 


993.14 


SAT, 75th percentile 


1212.83 


1219.02 


1219.23 


ACT, 25th percentile 


22.05 


20.86 


20.33 


ACT, 75th percentile 


26.29 


25.77 


25.31 



a The DCL sample includes students who participated in the 2005 and 2007 waves. This is the sample on which most of the reported analyses are based. 

b Integrated Postsecondary Education Data System (IPEDS) data include first-time, degree-seeking undergraduates and are weighted by enrollment. 

c Forthe IPEDS sample, this category includes American Indians, students of unknown backgrounds, and non-resident aliens. Forthe DCLsample,this category 
includes American Indians and any students who self-identified as "other" race/ethnicity. 



Collegiate Learning Assessment ( CL A) 
Student learning was assessed using 
the performance task of the Colle- 
giate Learning Assessment (CLA). 
The CLA aims to measure general 
skills-based competencies such as 
critical thinking, analytical reasoning, 
and written communication. 
Measures used to assess student 
learning consist of three sets of 
open-ended prompts that have been 



carefully constructed in consultation 
with experts on student assessment 
and elaborately pre-tested and piloted 
in prior work. The three components 
include: performance task, make an 
argument, and break an argument. 

We focus on the performance task 
because that component of the 
CLA was administered most 
uniformly across institutions, had 
the largest completion rate, and is 



the state-of-the-art component of 
the assessment instrument. The 
performance task allows students 
90 minutes to respond to a writing 
prompt that represents a “real- world” 
scenario in which they need to use a 
range of background documents 
(from memos and newspaper articles 
to reports, journal articles, and 
graphic representations) to solve 
a task or a dilemma. The testing 











materials, including the background 
documents, are accessed through 
a computer. For more 
information on the CLA, see 
www. collegiatelearningas se s sment. org. 

The Collegiate Learning Assessment 
was developed for instructional 
assessment and improvement, not for 
accountability purposes. Variation 
in CLA scores within individuals and 
the recent validity study organized 
by the Fund for the Improvement 
of Postsecondary Education (FIPSE) 
indicate that the CLA (as well as 
other currently available measures of 
learning in higher education) should 
not be used for high-stakes testing 
with consequences for individual 
students. Moreover, the CLA 
measures a select set of skills that 
represent a sub -set of skills taught 
in higher education. Although 
virtually all faculty agree that 
teaching critical thinking, analytical 
reasoning, and writing is at the core 
of undergraduate education, these 
skills do not capture the totality 
of student experiences or learning 
in specific subject areas and 
occupational fields. 

Analyses 

In addition to presenting descriptive 
results, we conducted multivariate 
regression analyses predicting 2007 
CLA performance task scores, 
while controlling for the 2005 CLA 



performance task scores. Moreover, 
for analyses of institutional selectivity, 
we use a three-level hierarchical linear 
model to estimate change in CLA 
scores over the full four years of 
college. It is worthwhile to note 
that our strategy differs from the 
value-added approach typically 
adopted by institutions using the 
CLA, which is also often based on 
cross-sectional data. Instead of 
estimating value-added models, 
we aim to understand the relationship 
between specific students’ character- 
istics and experiences and their 
growth on the CLA over time, and 
we use a longitudinal design to do so. 

Findings regarding college activities 
summarized in this report are based 
on regression analyses that control 
for students’ sociodemographic 
backgrounds (race/ ethnicity, gender, 
parental education and occupation, 
English home language, number of 
siblings, and two-parent household), 
high school characteristics (region, 
urbanicity, and racial composition [i.e., 
70 percent or more non- white]), 
academic preparation (number of 
Advanced Placement courses taken, 
high school GPA, and SAT/ACT per- 
formance), and institutions attended 
(using fixed effects). The three-level 
HLM models control for a similar set 
of background characteristics. 



Figures for predicted test scores are 
based on models with the complete 
set of covariates. In these predic- 
tions, all continuous variables (2005 
CLA scores, high school GPA, and 
SAT/ ACT performance) are set at 
their means. This implies that we are 
predicting 2007 CLA scores for an 
average student on those measures, 
i.e., a student with a mean 2005 CLA 
score, as well as mean high school 
GPA and SAT/ ACT performance. 
For categorical variables, our 
predictions are based on the 
reference category. For more 
information on statistical analyses, 
as well as robustness checks regarding 
ceiling and motivation effects, see 
the Methodological Appendix of 
Academically Adriji: Limited Learning 
on College Campuses (University of 
Chicago Press, 2011). 
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For more information, please visit http://highered.ssrc.org 




